Dropout Sampling for Robust Object Detection in Open-Set Conditions

Dropout Variational Inference, or Dropout Sampling, has been recently proposed as an approximation technique for Bayesian Deep Learning and evaluated for image classification and regression tasks. This paper investigates the utility of Dropout Sampling for object detection for the first time. We demonstrate how label uncertainty can be extracted from a state-of-the-art object detection system via Dropout Sampling. We evaluate this approach on a large synthetic dataset of 30,000 images, and a real-world dataset captured by a mobile robot in a versatile campus environment. We show that this uncertainty can be utilized to increase object detection performance under the open-set conditions that are typically encountered in robotic vision. A Dropout Sampling network is shown to achieve a 12.3% increase in recall (for the same precision score as a standard network) and a 15.1% increase in precision (for the same recall score as the standard network).Comment: to appear in IEEE International Conference on Robotics and Automation 2018 (ICRA 2018

Evaluating Merging Strategies for Sampling-based Uncertainty Techniques in Object Detection

There has been a recent emergence of sampling-based techniques for estimating epistemic uncertainty in deep neural networks. While these methods can be applied to classification or semantic segmentation tasks by simply averaging samples, this is not the case for object detection, where detection sample bounding boxes must be accurately associated and merged. A weak merging strategy can significantly degrade the performance of the detector and yield an unreliable uncertainty measure. This paper provides the first in-depth investigation of the effect of different association and merging strategies. We compare different combinations of three spatial and two semantic affinity measures with four clustering methods for MC Dropout with a Single Shot Multi-Box Detector. Our results show that the correct choice of affinity-clustering combination can greatly improve the effectiveness of the classification and spatial uncertainty estimation and the resulting object detection performance. We base our evaluation on a new mix of datasets that emulate near open-set conditions (semantically similar unknown classes), distant open-set conditions (semantically dissimilar unknown classes) and the common closed-set conditions (only known classes).Comment: to appear in IEEE International Conference on Robotics and Automation 2019 (ICRA 2019

Density-aware NeRF Ensembles: Quantifying Predictive Uncertainty in Neural Radiance Fields

We show that ensembling effectively quantifies model uncertainty in Neural Radiance Fields (NeRFs) if a density-aware epistemic uncertainty term is considered. The naive ensembles investigated in prior work simply average rendered RGB images to quantify the model uncertainty caused by conflicting explanations of the observed scene. In contrast, we additionally consider the termination probabilities along individual rays to identify epistemic model uncertainty due to a lack of knowledge about the parts of a scene unobserved during training. We achieve new state-of-the-art performance across established uncertainty quantification benchmarks for NeRFs, outperforming methods that require complex changes to the NeRF architecture and training regime. We furthermore demonstrate that NeRF uncertainty can be utilised for next-best view selection and model refinement

Uncertainty for Identifying Open-Set Errors in Visual Object Detection

Deployed into an open world, object detectors are prone to open-set errors, false positive detections of object classes not present in the training dataset. We propose GMM-Det, a real-time method for extracting epistemic uncertainty from object detectors to identify and reject open-set errors. GMM-Det trains the detector to produce a structured logit space that is modelled with class-specific Gaussian Mixture Models. At test time, open-set errors are identified by their low log-probability under all Gaussian Mixture Models. We test two common detector architectures, Faster R-CNN and RetinaNet, across three varied datasets spanning robotics and computer vision. Our results show that GMM-Det consistently outperforms existing uncertainty techniques for identifying and rejecting open-set detections, especially at the low-error-rate operating point required for safety-critical applications. GMM-Det maintains object detection performance, and introduces only minimal computational overhead. We also introduce a methodology for converting existing object detection datasets into specific open-set datasets to evaluate open-set performance in object detection

Class Anchor Clustering: a Loss for Distance-based Open Set Recognition

In open set recognition, deep neural networks encounter object classes that were unknown during training. Existing open set classifiers distinguish between known and unknown classes by measuring distance in a network's logit space, assuming that known classes cluster closer to the training data than unknown classes. However, this approach is applied post-hoc to networks trained with cross-entropy loss, which does not guarantee this clustering behaviour. To overcome this limitation, we introduce the Class Anchor Clustering (CAC) loss. CAC is a distance-based loss that explicitly trains known classes to form tight clusters around anchored class-dependent centres in the logit space. We show that training with CAC achieves state-of-the-art performance for distance-based open set classifiers on all six standard benchmark datasets, with a 15.2% AUROC increase on the challenging TinyImageNet, without sacrificing classification accuracy. We also show that our anchored class centres achieve higher open set performance than learnt class centres, particularly on object-based datasets and large numbers of training classes.Comment: Published at 2021 IEEE Winter Conference on Applications of Computer Vision (WACV

Uncertainty-Aware Lidar Place Recognition in Novel Environments

State-of-the-art approaches to lidar place recognition degrade significantly when tested on novel environments that are not present in their training dataset. To improve their reliability, we propose uncertainty-aware lidar place recognition, where each predicted place match must have an associated uncertainty that can be used to identify and reject potentially incorrect matches. We introduce a novel evaluation protocol designed to benchmark uncertainty-aware lidar place recognition, and present Deep Ensembles as the first uncertainty-aware approach for this task. Testing across three large-scale datasets and three state-of-the-art architectures, we show that Deep Ensembles consistently improves the performance of lidar place recognition in novel environments. Compared to a standard network, our results show that Deep Ensembles improves the Recall@1 by more than 5% and AuPR by more than 3% on average when tested on previously unseen environments. Our code repository will be made publicly available upon paper acceptance at https://github.com/csiro-robotics/Uncertainty-LPR.Comment: 7 pages, 4 figures. Under Revie

SAFE: Sensitivity-Aware Features for Out-of-Distribution Object Detection

We address the problem of out-of-distribution (OOD) detection for the task of object detection. We show that residual convolutional layers with batch normalisation produce Sensitivity-Aware FEatures (SAFE) that are consistently powerful for distinguishing in-distribution from out-of-distribution detections. By extracting SAFE vectors for every detected object, and training a multilayer perceptron on the surrogate task of distinguishing adversarially perturbed from clean in-distribution examples, we circumvent the need for realistic OOD training data, computationally expensive generative models, or retraining of the base object detector. SAFE outperforms the state-of-the-art OOD object detectors on multiple benchmarks by large margins, e.g. reducing the FPR95 by an absolute 30.6% from 48.3% to 17.7% on the OpenImages dataset

Probabilistic Object Detection: Definition and Evaluation

We introduce Probabilistic Object Detection, the task of detecting objects in images and accurately quantifying the spatial and semantic uncertainties of the detections. Given the lack of methods capable of assessing such probabilistic object detections, we present the new Probability-based Detection Quality measure (PDQ).Unlike AP-based measures, PDQ has no arbitrary thresholds and rewards spatial and label quality, and foreground/background separation quality while explicitly penalising false positive and false negative detections. We contrast PDQ with existing mAP and moLRP measures by evaluating state-of-the-art detectors and a Bayesian object detector based on Monte Carlo Dropout. Our experiments indicate that conventional object detectors tend to be spatially overconfident and thus perform poorly on the task of probabilistic object detection. Our paper aims to encourage the development of new object detection approaches that provide detections with accurately estimated spatial and label uncertainties and are of critical importance for deployment on robots and embodied AI systems in the real world.Comment: 21 pages, 25 figures, to appear in the proceedings of the winter conference on applications of computer vision WACV 202

Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019

Background: In an era of shifting global agendas and expanded emphasis on non-communicable diseases and injuries along with communicable diseases, sound evidence on trends by cause at the national level is essential. The Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) provides a systematic scientific assessment of published, publicly available, and contributed data on incidence, prevalence, and mortality for a mutually exclusive and collectively exhaustive list of diseases and injuries. Methods: GBD estimates incidence, prevalence, mortality, years of life lost (YLLs), years lived with disability (YLDs), and disability-adjusted life-years (DALYs) due to 369 diseases and injuries, for two sexes, and for 204 countries and territories. Input data were extracted from censuses, household surveys, civil registration and vital statistics, disease registries, health service use, air pollution monitors, satellite imaging, disease notifications, and other sources. Cause-specific death rates and cause fractions were calculated using the Cause of Death Ensemble model and spatiotemporal Gaussian process regression. Cause-specific deaths were adjusted to match the total all-cause deaths calculated as part of the GBD population, fertility, and mortality estimates. Deaths were multiplied by standard life expectancy at each age to calculate YLLs. A Bayesian meta-regression modelling tool, DisMod-MR 2.1, was used to ensure consistency between incidence, prevalence, remission, excess mortality, and cause-specific mortality for most causes. Prevalence estimates were multiplied by disability weights for mutually exclusive sequelae of diseases and injuries to calculate YLDs. We considered results in the context of the Socio-demographic Index (SDI), a composite indicator of income per capita, years of schooling, and fertility rate in females younger than 25 years. Uncertainty intervals (UIs) were generated for every metric using the 25th and 975th ordered 1000 draw values of the posterior distribution. Findings: Global health has steadily improved over the past 30 years as measured by age-standardised DALY rates. After taking into account population growth and ageing, the absolute number of DALYs has remained stable. Since 2010, the pace of decline in global age-standardised DALY rates has accelerated in age groups younger than 50 years compared with the 1990–2010 time period, with the greatest annualised rate of decline occurring in the 0–9-year age group. Six infectious diseases were among the top ten causes of DALYs in children younger than 10 years in 2019: lower respiratory infections (ranked second), diarrhoeal diseases (third), malaria (fifth), meningitis (sixth), whooping cough (ninth), and sexually transmitted infections (which, in this age group, is fully accounted for by congenital syphilis; ranked tenth). In adolescents aged 10–24 years, three injury causes were among the top causes of DALYs: road injuries (ranked first), self-harm (third), and interpersonal violence (fifth). Five of the causes that were in the top ten for ages 10–24 years were also in the top ten in the 25–49-year age group: road injuries (ranked first), HIV/AIDS (second), low back pain (fourth), headache disorders (fifth), and depressive disorders (sixth). In 2019, ischaemic heart disease and stroke were the top-ranked causes of DALYs in both the 50–74-year and 75-years-and-older age groups. Since 1990, there has been a marked shift towards a greater proportion of burden due to YLDs from non-communicable diseases and injuries. In 2019, there were 11 countries where non-communicable disease and injury YLDs constituted more than half of all disease burden. Decreases in age-standardised DALY rates have accelerated over the past decade in countries at the lower end of the SDI range, while improvements have started to stagnate or even reverse in countries with higher SDI. Interpretation: As disability becomes an increasingly large component of disease burden and a larger component of health expenditure, greater research and developm nt investment is needed to identify new, more effective intervention strategies. With a rapidly ageing global population, the demands on health services to deal with disabling outcomes, which increase with age, will require policy makers to anticipate these changes. The mix of universal and more geographically specific influences on health reinforces the need for regular reporting on population health in detail and by underlying cause to help decision makers to identify success stories of disease control to emulate, as well as opportunities to improve. Funding: Bill & Melinda Gates Foundation. © 2020 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 licens

Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980-2017 : a systematic analysis for the Global Burden of Disease Study 2017

Background Global development goals increasingly rely on country-specific estimates for benchmarking a nation's progress. To meet this need, the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2016 estimated global, regional, national, and, for selected locations, subnational cause-specific mortality beginning in the year 1980. Here we report an update to that study, making use of newly available data and improved methods. GBD 2017 provides a comprehensive assessment of cause-specific mortality for 282 causes in 195 countries and territories from 1980 to 2017. Methods The causes of death database is composed of vital registration (VR), verbal autopsy (VA), registry, survey, police, and surveillance data. GBD 2017 added ten VA studies, 127 country-years of VR data, 502 cancer-registry country-years, and an additional surveillance country-year. Expansions of the GBD cause of death hierarchy resulted in 18 additional causes estimated for GBD 2017. Newly available data led to subnational estimates for five additional countries Ethiopia, Iran, New Zealand, Norway, and Russia. Deaths assigned International Classification of Diseases (ICD) codes for non-specific, implausible, or intermediate causes of death were reassigned to underlying causes by redistribution algorithms that were incorporated into uncertainty estimation. We used statistical modelling tools developed for GBD, including the Cause of Death Ensemble model (CODErn), to generate cause fractions and cause specific death rates for each location, year, age, and sex. Instead of using UN estimates as in previous versions, GBD 2017 independently estimated population size and fertility rate for all locations. Years of life lost (YLLs) were then calculated as the sum of each death multiplied by the standard life expectancy at each age. All rates reported here are age-standardised. Findings At the broadest grouping of causes of death (Level 1), non-communicable diseases (NC Ds) comprised the greatest fraction of deaths, contributing to 73.4% (95% uncertainty interval [UI] 72.5-74.1) of total deaths in 2017, while communicable, maternal, neonatal, and nutritional (CMNN) causes accounted for 186% (17.9-19.6), and injuries 8.0% (7.7-8.2). Total numbers of deaths from NCD causes increased from 2007 to 2017 by 22.7% (21.5-23.9), representing an additional 7.61 million (7. 20-8.01) deaths estimated in 2017 versus 2007. The death rate from NCDs decreased globally by 7.9% (7.08.8). The number of deaths for CMNN causes decreased by 222% (20.0-24.0) and the death rate by 31.8% (30.1-33.3). Total deaths from injuries increased by 2.3% (0-5-4-0) between 2007 and 2017, and the death rate from injuries decreased by 13.7% (12.2-15.1) to 57.9 deaths (55.9-59.2) per 100 000 in 2017. Deaths from substance use disorders also increased, rising from 284 000 deaths (268 000-289 000) globally in 2007 to 352 000 (334 000-363 000) in 2017. Between 2007 and 2017, total deaths from conflict and terrorism increased by 118.0% (88.8-148.6). A greater reduction in total deaths and death rates was observed for some CMNN causes among children younger than 5 years than for older adults, such as a 36.4% (32.2-40.6) reduction in deaths from lower respiratory infections for children younger than 5 years compared with a 33.6% (31.2-36.1) increase in adults older than 70 years. Globally, the number of deaths was greater for men than for women at most ages in 2017, except at ages older than 85 years. Trends in global YLLs reflect an epidemiological transition, with decreases in total YLLs from enteric infections, respirator}, infections and tuberculosis, and maternal and neonatal disorders between 1990 and 2017; these were generally greater in magnitude at the lowest levels of the Socio-demographic Index (SDI). At the same time, there were large increases in YLLs from neoplasms and cardiovascular diseases. YLL rates decreased across the five leading Level 2 causes in all SDI quintiles. The leading causes of YLLs in 1990 neonatal disorders, lower respiratory infections, and diarrhoeal diseases were ranked second, fourth, and fifth, in 2017. Meanwhile, estimated YLLs increased for ischaemic heart disease (ranked first in 2017) and stroke (ranked third), even though YLL rates decreased. Population growth contributed to increased total deaths across the 20 leading Level 2 causes of mortality between 2007 and 2017. Decreases in the cause-specific mortality rate reduced the effect of population growth for all but three causes: substance use disorders, neurological disorders, and skin and subcutaneous diseases. Interpretation Improvements in global health have been unevenly distributed among populations. Deaths due to injuries, substance use disorders, armed conflict and terrorism, neoplasms, and cardiovascular disease are expanding threats to global health. For causes of death such as lower respiratory and enteric infections, more rapid progress occurred for children than for the oldest adults, and there is continuing disparity in mortality rates by sex across age groups. Reductions in the death rate of some common diseases are themselves slowing or have ceased, primarily for NCDs, and the death rate for selected causes has increased in the past decade. Copyright (C) 2018 The Author(s). Published by Elsevier Ltd.Peer reviewe